The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

The statistical heterogeneity of the non-independent and identically distributed (non-IID) data in local clients significantly limits the performance of federated learning. Previous attempts like FedProx, SCAFFOLD, MOON, FedNova and FedDyn resort to an optimization perspective, which requires an auxiliary term or re-weights local updates to calibrate the learning bias or the objective inconsistency. However, in addition to previous explorations for improvement in federated averaging, our analysis shows that another critical bottleneck is the poorer optima of client models in more heterogeneous conditions. We thus introduce a data-driven approach called FedSkip to improve the client optima by periodically skipping federated averaging and scattering local models to the cross devices. We provide theoretical analysis of the possible benefit from FedSkip and conduct extensive experiments on a range of datasets to demonstrate that FedSkip achieves much higher accuracy, better aggregation efficiency and competing communication efficiency. Source code is available at: https://github.com/MediaBrain-SJTU/FedSkip.

Purpose: The aim of this study was to demonstrate the utility of unsupervised domain adaptation (UDA) in automated knee osteoarthritis (OA) phenotype classification using a small dataset (n=50). Materials and Methods: For this retrospective study, we collected 3,166 three-dimensional (3D) double-echo steady-state magnetic resonance (MR) images from the Osteoarthritis Initiative dataset and 50 3D turbo/fast spin-echo MR images from our institute (in 2020 and 2021) as the source and target datasets, respectively. For each patient, the degree of knee OA was initially graded according to the MRI Osteoarthritis Knee Score (MOAKS) before being converted to binary OA phenotype labels. The proposed UDA pipeline included (a) pre-processing, which involved automatic segmentation and region-of-interest cropping; (b) source classifier training, which involved pre-training phenotype classifiers on the source dataset; (c) target encoder adaptation, which involved unsupervised adaption of the source encoder to the target encoder and (d) target classifier validation, which involved statistical analysis of the target classification performance evaluated by the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity and accuracy. Additionally, a classifier was trained without UDA for comparison. Results: The target classifier trained with UDA achieved improved AUROC, sensitivity, specificity and accuracy for both knee OA phenotypes compared with the classifier trained without UDA. Conclusion: The proposed UDA approach improves the performance of automated knee OA phenotype classification for small target datasets by utilising a large, high-quality source dataset for training. The results successfully demonstrated the advantages of the UDA approach in classification on small datasets.

Recently, the dominant DETR-based approaches apply central-concept spatial prior to accelerate Transformer detector convergency. These methods gradually refine the reference points to the center of target objects and imbue object queries with the updated central reference information for spatially conditional attention. However, centralizing reference points may severely deteriorate queries' saliency and confuse detectors due to the indiscriminative spatial prior. To bridge the gap between the reference points of salient queries and Transformer detectors, we propose SAlient Point-based DETR (SAP-DETR) by treating object detection as a transformation from salient points to instance objects. In SAP-DETR, we explicitly initialize a query-specific reference point for each object query, gradually aggregate them into an instance object, and then predict the distance from each side of the bounding box to these points. By rapidly attending to query-specific reference region and other conditional extreme regions from the image features, SAP-DETR can effectively bridge the gap between the salient point and the query-based Transformer detector with a significant convergency speed. Our extensive experiments have demonstrated that SAP-DETR achieves 1.4 times convergency speed with competitive performance. Under the standard training scheme, SAP-DETR stably promotes the SOTA approaches by 1.0 AP. Based on ResNet-DC-101, SAP-DETR achieves 46.9 AP.

在许多现实世界中的机器学习应用中，亚种群的转移存在着极大地存在，指的是包含相同亚种群组的培训和测试分布，但在亚种群频率中有所不同。重要性重新加权是通过对训练数据集中每个样本施加恒定或自适应抽样权重来处理亚种群转移问题的正常方法。但是，最近的一些研究已经认识到，这些方法中的大多数无法改善性能，而不是经验风险最小化，尤其是当应用于过度参数化的神经网络时。在这项工作中，我们提出了一个简单而实用的框架，称为“不确定性感知混合”（UMIX），以根据样品不确定性重新加权“混合”样品来减轻过度参数化模型中的过度拟合问题。基于训练 - 注射器的不确定性估计为每个样品的拟议UMIX配备，以灵活地表征亚群分布。我们还提供有见地的理论分析，以验证UMIX是否在先前的工作中实现了更好的概括界限。此外，我们在广泛的任务上进行了广泛的经验研究，以验证我们方法的有效性，既有定性和定量。

自我介绍在训练过程中利用自身的非均匀软监管，并在没有任何运行时成本的情况下提高性能。但是，在训练过程中的开销经常被忽略，但是在巨型模型的时代，培训期间的时间和记忆开销越来越重要。本文提出了一种名为ZIPF标签平滑（ZIPF的LS）的有效自我验证方法，该方法使用网络的直立预测来生成软监管，该软监管在不使用任何对比样本或辅助参数的情况下符合ZIPF分布。我们的想法来自经验观察，即当对网络进行适当训练时，在按样品的大小和平均分类后，应遵循分布的分布，让人联想到ZIPF的自然语言频率统计信息，这是在按样品中的大小和平均值进行排序之后进行的。 。通过在样本级别和整个培训期内强制执行此属性，我们发现预测准确性可以大大提高。使用INAT21细粒分类数据集上的RESNET50，与香草基线相比，我们的技术获得了 +3.61％的准确性增长，而与先前的标签平滑或自我验证策略相比，增益增加了0.88％。该实现可在https://github.com/megvii-research/zipfls上公开获得。

整个幻灯片图像（WSI）分类通常依赖于深度监督的多个实例学习（MIL）方法来处理Gigapixel分辨率图像和幻灯片级标签。然而，深度学习的不错的表现来自利用大量数据集和不同的样本，敦促需要有效的培训管道来扩展到大型数据集和数据增强技术以进行多元化样品。但是，当前基于MIL的WSI分类管道是内存量的且计算的，因为它们通常组装成千上万的补丁作为计算袋。另一方面，尽管它们在其他任务中很受欢迎，但对于WSI MIL Frameworks来说，数据增强尚未探索。为了解决它们，我们提出了Remix，这是基于MIL WSI分类的一般有效框架。它包括两个步骤：减少和混合。首先，它通过用实例原型（即贴片群质心）代替实例，从而减少了WSI袋中的实例数量。然后，我们提出了一个``混合式''增强，其中包含四个在线，随机和灵活的潜在空间扩展。它带来了潜在空间的多样化和可靠的班级身份的语义变化，同时实施语义扰动不变性。我们通过两种最先进的MIL方法在两个公共数据集上评估混音。在我们的实验中，已经实现了精确度，准确性和召回率的一致提高，但随着训练时间和记忆消耗的减少阶段，它表明了混音的有效性和效率。代码可用。

深度神经网络通常使用遇到数量不平衡和分类难度不平衡问题的数据集的性能很差。尽管在该领域取得了进展，但现有的两阶段方法中仍然存在数据集偏差或域转移问题。因此，提出了一个分阶段的渐进学习时间表，从而提出了从表示学习到上层分类器培训的平稳转移。这对严重失衡或较小尺度的数据集具有更大的有效性。设计了耦合 - 调节损失损失函数，耦合校正项，局灶性损失和LDAM损失。损失可以更好地处理数量不平衡和异常值，同时调节具有不同分类困难的样本的注意力重点。这些方法在多个基准数据集上取得了令人满意的结果，包括不平衡的CIFAR10，不平衡的CIFAR100，Imagenet-LT和Inaturalist 2018，并且还可以轻松地将其用于其他不平衡分类模型。

Vision-Language预培训是一个新兴和快速发展的研究主题，将多模态知识从丰富的资源预训练任务转移到有限资源下游任务。与主要学习单个通用编码器的现有作品不同，我们提出了一种可训练的通用编码器 - 解码器网络（UNI-EDEN），以促进视觉语言感知（例如，视觉问题应答）和生成（例如，图像标题）。 UNI-EDEN是一种基于双流变换器的结构，由三个模块组成：对象和句子编码器，其单独了解每个模态的表示，以及通过模态交互能够实现多模态推理和句子的句子解码器。考虑到每个图像的语言表示可以跨越该层次结构的不同粒度，包括从简单到全面，个人标签，短语和自然句子，我们通过多粒愿景语言代理任务预先列车UNI-EDEN：屏蔽对象分类（MOC），蒙版区域短语生成（MRPG），图像句匹配（ISM）和屏蔽句生成（MSG）。以这种方式，UNI-EDEN赋予了多模态表示提取和语言建模的功率。广泛的实验证明了通过微调到四个视觉语言感知和发电下游任务来展示Uni-Eden的概括性。

域适应（DA）最近在医学影像社区提出了强烈的兴趣。虽然已经提出了大量DA技术进行了用于图像分割，但大多数这些技术已经在私有数据集或小公共可用数据集上验证。此外，这些数据集主要解决了单级问题。为了解决这些限制，与第24届医学图像计算和计算机辅助干预（Miccai 2021）结合第24届国际会议组织交叉模态域适应（Crossmoda）挑战。 Crossmoda是无监督跨型号DA的第一个大型和多级基准。挑战的目标是分割参与前庭施瓦新瘤（VS）的后续和治疗规划的两个关键脑结构：VS和Cochleas。目前，使用对比度增强的T1（CET1）MRI进行VS患者的诊断和监测。然而，使用诸如高分辨率T2（HRT2）MRI的非对比度序列越来越感兴趣。因此，我们创建了一个无人监督的跨模型分段基准。训练集提供注释CET1（n = 105）和未配对的非注释的HRT2（n = 105）。目的是在测试集中提供的HRT2上自动对HRT2进行单侧VS和双侧耳蜗分割（n = 137）。共有16支球队提交了评估阶段的算法。顶级履行团队达成的表现水平非常高（最佳中位数骰子 - vs：88.4％; Cochleas：85.7％）并接近完全监督（中位数骰子 - vs：92.5％;耳蜗：87.7％）。所有顶级执行方法都使用图像到图像转换方法将源域图像转换为伪目标域图像。然后使用这些生成的图像和为源图像提供的手动注释进行培训分割网络。